Tiny datablock in saving Hadoop distributed file system wasted memory
نویسندگان
چکیده
<p>Hadoop distributed file system (HDFS) is the whereby Hadoop use it to store all upcoming data inside it. Since been declared, HDFS consuming a huge memory amount in order serve normal dataset. Nonetheless, current saving mechanism save only one datablock. Thus, with just 5 Mb size will take up whole datablock capacity causing rest of unavailable for other files, and this considered waste serving This paper proposed method called tiny datablock-HDFS (TD-HDFS) increase usability hosting capabilities by reducing minimum capacity, then merging related datablocks into master consists virtual that contain small files together; exploit full The result study running wasted same read/write performance. results were examined through comparison between standard solution study.</p><textarea id="BFI_DATA" style="width: 1px; height: display: none;"></textarea><textarea none;"></textarea><div id="WidgetFloaterPanels" class="LTRStyle" style="display: none; text-align: left; direction: ltr; visibility: hidden;"><div id="WidgetFloater" none;" onmouseover="Microsoft.Translator.OnMouseOverFloater()" onmouseout="Microsoft.Translator.OnMouseOutFloater()"><div id="WidgetLogoPanel"><span id="WidgetTranslateWithSpan"><span>TRANSLATE </span><img id="FloaterLogo" alt="" /></span> <span id="WidgetCloseButton" title="Exit Translation" onclick="Microsoft.Translator.FloaterOnClose()">x</span></div><div id="LanguageMenuPanel"><div class="DDStyle_outer"><input id="LanguageMenu_svid" onclick="this.select()" type="text" name="LanguageMenu_svid" value="en" /> <input id="LanguageMenu_textid" name="LanguageMenu_textid" id="__LanguageMenu_header" class="DDStyle" onclick="return LanguageMenu &amp;&amp; !LanguageMenu.Show('__LanguageMenu_popup', event);" onkeydown="return event);">English</span><div style="position: relative; left: 0;"><div absolute; ;left: 0px;"><div id="__LanguageMenu_popup" none;"><table id="LanguageMenu" border="0"><tbody><tr><td><a LanguageMenu.onclick('ar');" tabindex="-1" href="#ar">Arabic</a></td><td><a LanguageMenu.onclick('he');" href="#he">Hebrew</a></td><td><a LanguageMenu.onclick('pl');" href="#pl">Polish</a></td></tr><tr><td><a LanguageMenu.onclick('bg');" href="#bg">Bulgarian</a></td><td><a LanguageMenu.onclick('hi');" href="#hi">Hindi</a></td><td><a LanguageMenu.onclick('pt');" href="#pt">Portuguese</a></td></tr><tr><td><a LanguageMenu.onclick('ca');" href="#ca">Catalan</a></td><td><a LanguageMenu.onclick('mww');" href="#mww">Hmong Daw</a></td><td><a LanguageMenu.onclick('ro');" href="#ro">Romanian</a></td></tr><tr><td><a LanguageMenu.onclick('zh-CHS');" href="#zh-CHS">Chinese Simplified</a></td><td><a LanguageMenu.onclick('hu');" href="#hu">Hungarian</a></td><td><a LanguageMenu.onclick('ru');" href="#ru">Russian</a></td></tr><tr><td><a LanguageMenu.onclick('zh-CHT');" href="#zh-CHT">Chinese Traditional</a></td><td><a LanguageMenu.onclick('id');" href="#id">Indonesian</a></td><td><a LanguageMenu.onclick('sk');" href="#sk">Slovak</a></td></tr><tr><td><a LanguageMenu.onclick('cs');" href="#cs">Czech</a></td><td><a LanguageMenu.onclick('it');" href="#it">Italian</a></td><td><a LanguageMenu.onclick('sl');" href="#sl">Slovenian</a></td></tr><tr><td><a LanguageMenu.onclick('da');" href="#da">Danish</a></td><td><a LanguageMenu.onclick('ja');" href="#ja">Japanese</a></td><td><a LanguageMenu.onclick('es');" href="#es">Spanish</a></td></tr><tr><td><a LanguageMenu.onclick('nl');" href="#nl">Dutch</a></td><td><a LanguageMenu.onclick('tlh');" href="#tlh">Klingon</a></td><td><a LanguageMenu.onclick('sv');" href="#sv">Swedish</a></td></tr><tr><td><a LanguageMenu.onclick('en');" href="#en">English</a></td><td><a LanguageMenu.onclick('ko');" href="#ko">Korean</a></td><td><a LanguageMenu.onclick('th');" href="#th">Thai</a></td></tr><tr><td><a LanguageMenu.onclick('et');" href="#et">Estonian</a></td><td><a LanguageMenu.onclick('lv');" href="#lv">Latvian</a></td><td><a LanguageMenu.onclick('tr');" href="#tr">Turkish</a></td></tr><tr><td><a LanguageMenu.onclick('fi');" href="#fi">Finnish</a></td><td><a LanguageMenu.onclick('lt');" href="#lt">Lithuanian</a></td><td><a LanguageMenu.onclick('uk');" href="#uk">Ukrainian</a></td></tr><tr><td><a LanguageMenu.onclick('fr');" href="#fr">French</a></td><td><a LanguageMenu.onclick('ms');" href="#ms">Malay</a></td><td><a LanguageMenu.onclick('ur');" href="#ur">Urdu</a></td></tr><tr><td><a LanguageMenu.onclick('de');" href="#de">German</a></td><td><a LanguageMenu.onclick('mt');" href="#mt">Maltese</a></td><td><a LanguageMenu.onclick('vi');" href="#vi">Vietnamese</a></td></tr><tr><td><a LanguageMenu.onclick('el');" href="#el">Greek</a></td><td><a LanguageMenu.onclick('no');" href="#no">Norwegian</a></td><td><a LanguageMenu.onclick('cy');" href="#cy">Welsh</a></td></tr><tr><td><a LanguageMenu.onclick('ht');" href="#ht">Haitian Creole</a></td><td><a LanguageMenu.onclick('fa');" href="#fa">Persian</a></td><td> </td></tr></tbody></table><img style="height: 7px; width: 17px; border-width: 0px; 20px;" /></div></div></div></div><script type="text/javascript">// <![CDATA[
 var LanguageMenu; LanguageMenu_keys=["ar","bg","ca","zh-CHS","zh-CHT","cs","da","nl","en","et","fi","fr","de","el","ht","he","hi","mww","hu","id","it","ja","tlh","ko","lv","lt","ms","mt","no","fa","pl","pt","ro","ru","sk","sl","es","sv","th","tr","uk","ur","vi","cy"]; LanguageMenu_values=["Arabic","Bulgarian","Catalan","Chinese Simplified","Chinese Traditional","Czech","Danish","Dutch","English","Estonian","Finnish","French","German","Greek","Haitian Creole","Hebrew","Hindi","Hmong Daw","Hungarian","Indonesian","Italian","Japanese","Klingon","Korean","Latvian","Lithuanian","Malay","Maltese","Norwegian","Persian","Polish","Portuguese","Romanian","Russian","Slovak","Slovenian","Spanish","Swedish","Thai","Turkish","Ukrainian","Urdu","Vietnamese","Welsh"]; LanguageMenu_callback=function(){ }; LanguageMenu_popupid='__LanguageMenu_popup';
 // ]]></script></div><div id="CTFLinksPanel"><span id="ExternalLinksPanel"><a id="HelpLink" title="Help" href="https://go.microsoft.com/?linkid=9722454" target="_blank"> <img id="HelpImg" /></a> <a id="EmbedLink" title="Get widget your own site" href="javascript:Microsoft.Translator.FloaterShowEmbed()"> id="EmbedImg" id="ShareLink" title="Share translated page friends" href="javascript:Microsoft.Translator.FloaterShowSharePanel()"> id="ShareImg" </span></div><div id="FloaterProgressBar"> </div></div><div id="WidgetFloaterCollapsed" onmouseover="Microsoft.Translator.OnMouseOverFloater()"><span>TRANSLATE id="CollapsedLogoImg" /></div><div id="FloaterSharePanel" none;"><div id="ShareTextDiv"><span id="ShareTextSpan"> COPY THE URL BELOW id="ShareTextboxDiv"><input id="ShareTextbox" name="ShareTextbox" readonly="readonly" <!--a id="TwitterLink" on Twitter"> id="TwitterImg" <a-- id="FacebookLink" Facebook"> id="FacebookImg" /></a--> id="EmailLink" title="Email translation"> id="EmailImg" /></a></div><div id="ShareFooter"><span id="ShareHelpSpan"><a id="ShareHelpLink"> id="ShareHelpImg" /></a></span> id="ShareBackSpan"><a id="ShareBack" title="Back To href="javascript:Microsoft.Translator.FloaterOnShareBackClick()"> Back</a></span></div><input id="EmailSubject" type="hidden" name="EmailSubject" value="Check out {0} from {1}" id="EmailBody" name="EmailBody" value="Translated: {0}%0d%0aOriginal: {1}%0d%0a%0d%0aAutomatic translation powered Microsoft® Translator%0d%0ahttp://www.bing.com/translator?ref=MSTWidget" id="ShareHelpText" value="This link allows visitors launch automatically translate {0}." id="FloaterEmbed" id="EmbedTextDiv"><span id="EmbedTextSpan">EMBED SNIPPET IN YOUR SITE</span> id="EmbedHelpLink" title="Copy code place HTML."> id="EmbedHelpImg" id="EmbedTextboxDiv"><input id="EmbedSnippetTextBox" name="EmbedSnippetTextBox" value="&lt;div id='MicrosoftTranslatorWidget' class='Dark' style='color:white;background-color:#555555'&gt;&lt;/div&gt;&lt;script type='text/javascript'&gt;setTimeout(function(){var s=document.createElement('script');s.type='text/javascript';s.charset='UTF-8';s.src=((location location.href location.href.indexOf('https') == 0)?'https://ssl.microsofttranslator.com':'http://www.microsofttranslator.com')+'/ajax/v3/WidgetV3.ashx?siteData=ueOIGRSKkd965FeEGM5JtQ**&amp;ctf=true&amp;ui=true&amp;settings=manual&amp;from=en';var p=document.getElementsByTagName('head')[0]||document.documentElement;p.insertBefore(s,p.firstChild); },0);&lt;/script&gt;" id="EmbedNoticeDiv"><span id="EmbedNoticeSpan">Enable collaborative features customize widget: href="http://www.bing.com/widget/translator" target="_blank">Bing Webmaster Portal</a></span></div><div id="EmbedFooterDiv"><span id="EmbedBackSpan"><a href="javascript:Microsoft.Translator.FloaterOnEmbedBackClick()">Back</a></span></div></div><script intervalId = setInterval(function () { if (MtPopUpList) new MtPopUpList(); langMenu document.getElementById(LanguageMenu_popupid); origLangDiv document.createElement("div"); origLangDiv.id "OriginalLanguageDiv"; origLangDiv.innerHTML "<span id='OriginalTextSpan'>ORIGINAL: </span><span id='OriginalLanguageSpan'></span>"; langMenu.appendChild(origLangDiv); LanguageMenu.Init('LanguageMenu', LanguageMenu_keys, LanguageMenu_values, LanguageMenu_callback, LanguageMenu_popupid); window["LanguageMenu"] clearInterval(intervalId); } }, 1);
 ]]></script></div>
منابع مشابه
Snapshots in Hadoop Distributed File System
The ability to take snapshots is an essential functionality of any file system, as snapshots enable system administrators to perform data backup and recovery in case of failure. We present a low-overhead snapshot solution for HDFS, a popular distributed file system for large clusters of commodity servers. Our solution obviates the need for complex distributed snapshot algorithms, by taking adva...
متن کاملGoogle File System and Hadoop Distributed File System - An Analogy
Big Data has indeed been the word which IT Industry is talking about lately. With advancement of automation and data being processed in real time, it has now become a necessity for companies to look forward to sustainable solutions to store their huge datasets and compute valuable information out of it. High performance computing heavily relies on distributed environments to process large chunk...
متن کاملThe Hadoop Distributed File System: Balancing Portabilty
Hadoop is a software framework that supports data intensive distributed application. Hadoop creates clusters of machine and coordinates the work among them. It include two major component, HDFS (Hadoop Distributed File System) and MapReduce. HDFS is designed to store large amount of data reliably and provide high availability of data to user application running at client. It creates multiple da...
متن کاملQoS-Aware Data Replication in Hadoop Distributed File System
Dr. Sunita Varma Department of ComputerTechnology and Application S. G. S. I. T. S. Indore, (M. P.), India [email protected] Ms. Gopi Khatri Department of Computer Engineering S. G. S. I. T. S Indore, (M. P.), India [email protected] --------------------------------------------------------------------ABSTRACT------------------------------------------------------------Cloud computin...
متن کاملذخیره در منابع من
با ذخیره ی این منبع در منابع من، دسترسی به آن را برای استفاده های بعدی آسان تر کنید
ژورنال
عنوان ژورنال: International Journal of Electrical and Computer Engineering
سال: 2023
ISSN: ['2088-8708']
DOI: https://doi.org/10.11591/ijece.v13i2.pp1757-1772